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Abstract A measure of dependence is said to be equitable if it gives similar scores to equally noisy 
relationship of different types. In practice, we do not know what kind of functional relationship is underlying 
two given observations, Hence the equitability of dependence measure is critical in analysis and by scoring 
relationships according to an equitable measure one hopes to find important patterns of any type of further 
examination. In this paper, we introduce our definition of equitability of a dependence measure, which is 
naturally from this initial description, and Further more power-equitable(weak-equitable) is introduced which 
is of the most practical meaning in evaluating the equitablity of a dependence measure. 
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1 Introduction 

Measuring dependence between two variables or sequences plays a fundamental role in various kind of data 
analysis, such as fMRI data, genetic data. How should one quantify such a dependence without bias for 
relationship of a specific form? This gives rise to the concept ’’Equitability” of a dependence measure [TO] . 
[36] . By scoring relationships according to an equitable measure one hopes to find important patterns of any 
type further examination. The first description of ’’Equitability” given in m is ” A measure of dependence 
is said to be equitable if it gives similar scores to equally noisy relationship of different types”. In [37], 
authors pointed out that there is no dependence satisfying the definition of equitability given in m, the 
R 2 — Equitability. Furthermore, they give a definition of ” self-equitability”, and show by theoretically proof, 
and numerical simulation that mutual information satisfies the ” self-equitability”. 

Equitability of dependence measure means it is equitable to all kinds of functional relationship, as the 
description goes, give similar score to equally noisy functional relationship. Taking correlation for example, it 
is un-equitable, since it gives very small scores to nonlinear relationships that can not be well approximated 
by linear function. There are two key terms in this sentence ’’equally noisy relationship” and ’’similar score”. 
The noise level is given by 1 — R 2 {f(x), y} for the model y = f(x) + e in {36], and [37] . However, in their 
definition of R 2 —Equitability there is no words related to ’’equally noisy relationship”. In this paper, we 
introduce the model signal-to-noise ratio (MSNR)and signal signal-to-noise ratio (SSNR) to control the noise 
level in our simulation, and give the definition of Equitability in a more natural way, which, at the same 
time, leads to the definition of weak-equitable. 

The self-equitable defined and discussed in [37] focused on the equitability of dependence measure when 
it is regarded as a measure of noise in the data set. If a dependence measure is self-equitable, then it does 
not depend on what the specific functional relationship between x and y would be in the absence of noise. 
However, the noise in the data set is difficult to measure, or no one can tell though the methods existed 
how much noise is in the data set. In other words, self-equitable is necessary for a dependence measure used 
as a noise measure. However, in practice, people usually used the dependence measure as a test statistic, 
we say x and y are significantly associated with each other when the value D[x;y] obtained by dependence 
measure D is larger than a given threshold, A , with significant value a, which motivated us to the definition 
of power-equitable, that is, weak-equitable. 

In this paper, we try to make clear the statement debates in [35] that MIC is more equitable that MI. 
In addition, we introduce a new definition, weak-equitable (power-equitable), which is meaningful when a 
dependence measure is used in independent test problem. In the next section, we will recall firstly the 
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definitions given in m, and then introduce the definition of equitable, and weak-equitable. Simulation 
results are given in section 3. 


2 Definitions of Equitability 

A measure of dependence is said to be equitable if it gives similar scores to equally noisy relationships of 
different types .10,36. In other words, a measure of how much noise is in an x-y scatter plot should 
not depend on what the specific functional relationship between x and y would be in the absence of noise 
Ezj. Justin B. Kinney etal. gave the definition of R 2 — equitable and Self-equitable [37]. They pointed out 
that the dependence measure satisfying R 2 —equitable does not exist, and dependence measure satisfying 
Self-equitability exists, mutual information is one of them. 

Here we recall the definition of Self-equitable and i? 2 -equitable. 

Definition 2.1 A dependence measure D[X;Y] is R 2 -equitable if and only if, when evaluated on a joint 
probability distribution P(X,Y), that corresponds to a noisy functional relationship between two real random 
variables X and Y, the following relation holds: 

D[X;Y] = g(R 2 [f(X);Y]) (1) 

Here, g is a function that does not depend on P(X, Y) and f is the function defining the noisy functional 
relationship, i.e., Y = f{X) + y, for some random variable y. The noise term ?y may depend on as long as ? 
has no additional dependence on X, i.e, x —> f(x) —» y forms a Markov chain. 

Definition 2.2 A dependence measure D[x~,y\ is self-equitable if and only if 

D[x] y] = D[f(x); y] (2) 

whenever f is a deterministic function and x —> /( x) —> y forms a Markov chain. 

In the definition of i? 2 -equitable, the term ’’gives similar scores to equally noisy relationships of different 
types” is described by the dependence measure as a function, independent with x and y, of noise, so the 
meaning of equitability is conveyed implicitly. Differently, our definition of equitable conveys the meaning 
,” equally noisy relationship ”, directly through a mathematical definition of noisy-equal model based on the 
model signal-to-noise ratio (MSNR). 

Definition 2.3 (MSNR) Signal-to-Noise Ration(SNR) for a model, y = f(x) + e, is given by 

MSNRM) = ^ (3) 

var(£) 


where var(x) is the variance of x. 

Remark 1: 

1. Two models, y\ = f\(x) + £i and j /2 = fi{x) +£2 with the same MSNR is called noisy-equal models. 

2. In [ID], [37], the noise level is measured by 1 -i? 2 (/(x), /( x) + e) = 1 — p 2 (f(x), f{x) + e), theoretically, 
we have 


R 2 i.f{.x),f{x)+£) = 


1 


A 


(4) 


MSNR e (f ) 


3. How to get noisy-equal models? Given two models, y\ = fi(x) +£1 and 2/2 = f 2 (x) +£ 2 , we set £1 = £, 
£2 = |, where a 2 = ^(/Ra?)) ; £ independent with x, then 
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MSNR ei (f 1 ) 


var(yi) var(fi(x)) + var(e) 
var{e) var{e) 

var(fi(x)) /a 2 + var(e/a) 
var(e/a) 


var(y2) 

var(e 2 ) 


MSNR £2 (f 2 ) 


(5) 


Hence, we get noisy-equal models y ± = f^x) + £ and y 2 = f 2 (x) + §, where a 2 = ^(/ato) - 

However, when mean value is not zero, MSNR e is not equal to SNR, we need to go back to the definition 
of real SNR, which is SNR s (f) = . In most of the case, especially in electronic engineers, it equal 

to the signal square to noise square ration. 


Definition 2.4 (SSNR) SNR for signals, y = f{x) + e, is given by 

SSNR E (f) = \ 

Remark 2: 


( 6 ) 


1. Two signals, y\ = fi(x) + eq and y 2 = f 2 {x) + e 2 with the same SSNR is called noisy-equal signals. 

2. We use a heuristic algorithm to get noisy-equal signals. Given two signals, y-\ = fi(x) + £1 and 
y -2 = fiix) + £ 2 , calculate the SSNR of y 1; use it as optimization objective. Then we generate 
random e 2 , and let SSNR of y 2 approximate SSNR of y-\. 


Algorithm 1: The heuristic algorithm for generating noisy-equal signals 


1 snr . o . r <— function ( sig , noise ) { 

2 return (crossprod ( sig , sig ) / 

crossprod ( noise , noise)) 

3 } 

4 

5 guessNoise . r<— function ( sig , snr , 

steps = 100 , t hr = 0.03){ 
e crossprod _ noise<— crossprod ( sig , 
sig)/snr 

7 n<— length ( sig ) 

s rgnoise<— rep ( 0 ,n) 

9 step <—0 

10 delta<—9999 

11 an<—( crossprod _noise/n) '0.5 

12 mxan<— an 

13 m i an<—mxan *—1 

ii snr . r<— 0 

15 repeatj 

is mean. noise<— runif (1 ,min=mian, 

max=mxan) 

17 sd_noise<— ( crossprod _ noise/n— 


mean.noise '2) '0.5 

is gnoise<—rnorm(n ,mean_noise ,sd_ 

noise) 

is snr . r<—snr . o ( sig , gnoise ) 

20 step<—step+1 

21 tdelta<— abs( snr . r—snr ) 

22 if (tdelta<=delta ) { 

23 r g n o i s e<— gnoise 

24 delta<—t delta 

25 } 

26 #gc () 

27 i f ( abs (snr . r—snr )<=t hr | | step> 

steps ) { 

28 break 

29 } 

30 } 

31 #print ( delta) 

32 return ( list ( noise=rgnoise , snr=snr 

r )) 

33 } 


Then, we fix n-1 components of e 2 , we get 


E (h(Xi) + £2i) 2 + (f2(x n ) + £2n) 2 

SSNReifi) = ^^- 

E4 + 4 


(7) 
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Because SSNR e (fi), / 2 (xi),i = 1, ...,n and e 2 i,i = 1, ...,n — 1 are all known, we can solve the eq{7j 
and generate accurate noise to satisfy the SSNR. 

Definition 2.5 (Equitable) A dependence measure D[x;y\ is Equitable if and only if 

D[x-,y 1 }= D[x-,y 2 ] (8) 

for any noisy-equal models y\ = fi(x) + £± and y 2 = / 2 (;r) + e 2 . 

This definition is theoretically equal to that of i? 2 -equitable presented in 133 > but from a different point 
of view, which is more naturally and heuristically. Further, it helps us to look great insight into equitabilitiy 
of dependence measures, seen in the following section. 

The self-equitable focused on the equitability of dependence measure when it is regarded as a measure of 
noise in the data set. If a dependence measure is self-equitable, then it does not depend on what the specific 
functional relationship between x and y would be in the absence of noise. However, the noise in the data set is 
difficult to measure, or no one can tell though the methods existed how much noise is in the data set. In other 
words, self-equitable is necessary for a dependence measure used as a noise measure. However, in practice, 
people usually used the dependence measure as a test statistic, we say x and y are significantly associated 
with each other when the value D[x ; y\ obtained by dependence measure D is larger than a given threshold, 
A , with significant value a, which motivated us to the definition of power-equitable, that is, weak-equitable. 
To be formally, 

Definition 2.6 (Power-equitable) A dependence measure D[x;y] is Power-equitable if and only if 

Power[f\} = Power [/ 2 ] (9) 

for any noisy-equal models y\ = fi(x) + E\ and ?/ 2 = / 2 (x) + e 2; where Power[f] is the testing power of 
dependence measure D for detecting functional relationship f underlying noisy data set. 

So, if a dependence measure is equitable, then it is power-equitable, but the converse is not right, and we 
refer the power-equitable as weak-equitable. 

Weak-equitability can be explained as that: if a dependence measure, D, is weak-equitable, then all kinds 
of functional relationship underlying equally noisy data sets will be detected with the same possibility by D 
with significant level a, which is quite meaningful and interesting. This allowed, similar to a equitable case, 
us to use D as a test to determine whether or not x and y is significantly associated with each other. As 
mentioned before, self-equitable is meaningful, when D is used as a noise measure. 


3 Simulation Results 

In this section, we discuss the self-equitability, equitability, and power-equitability of the most popular 
dependence measures: MIC [TO]. CDC [35], RPC [TO], HHGjS], Pearson Correlation coefficient (pcor), Spear¬ 
man’s Rank Correlation(scor), Kentall’s r(kcor), curve correlation [13] . HSIC [27], normalized mutual infor¬ 
mational) jTOj. A detailed discussion of these measures can found in [35] . Although the Equitability of 
dcor [4] is discussed in [36], we also discuss it. In [36], authors pointed that there is a trade-off between 
equitability and test power of MIC. We will also discuss it in the following section. 

3.1 Equitability 

In this part, we analysis the equitability of MIC, CDC, RDC, HHG, pcor, scor, kcor, dcor, HSIC and MI. 
Although, there is some theoretical results given in 133 showing that dependence measure satisfying the 
definition of equitable do not exist, we give a detailed simulation results to further explanation. 

In our simulation, each time x is sampled from uniform distribution with length 5000, and noise is drawn 
from standard normal distribution, the MSNR is controlled by using ([5|, the SSNR is controlled by the 
heuristic algorithm, then we get one value of D(x, y = f(x) + e). We do this 100 times for each kind of 
functional types defined in appendix. 
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In Figu re [Tj Equitability of ACE, and RDC is analyzed. According to the definition of equitability 
(Definition 2.6), if we get almost the same score given by dependence measure D to each kind of functional 
type, then D is equitable. In our simulation, it is equal to that if the lines with different colors in the figure 
are consistent, then D is equitable. From the results presented, we can see that both RDC and ACE are not 
equitable. 

In Figure[2j Equitability of MIC, MI, CDC, poor, kcor, scor is analyzed, and all of them are not equitable. 
Interestingly, MI is equitable in a small set of functional types except four types: Sigmoid, Lopsided L-shaped, 
L-shaped and Spike. MI is more equitable than MIC, since MI values of 17 functional types (except the four 
types mentioned before) are centered around 0.8 with smaller variance than that of MIC. 

We can find that dependence measures in Figure [3] are not as stable as that of Figure [l] and Figure [2j 
However the sd(SSNR) and range(SSNR) are not larger than sd(MSNR) and range(MSNR) correspondingly. 
It because the heuristic algorithm 1 and eq. [7] used by SSNR simulations random generate the noise £ 2 , but 
eq. [5] generate the noise £2 based on e\. 


3.2 Weak-Equitable 

In this part, we discuss the weak-equitable of these dependence measures. In our simulation, we set MSNR 
ranging from 0.7509836 to 3.0057, because high power in models with large MSNR is basic requirement for 
dependence measure using in independent test, and it is inevitable that for all dependence measure it will 
have a very lower power on heavy noisy models (models with very small MSNR). The results are given in 
Figure [ij Figure [5j and Figure [6] 

From the results presented in the following figures, we know that HHG is almost power-equitable, but 
to get its value is time-consuming in large data sets, ACE is secondary to HHG. But ACE is sensitive to 
outliers, so, we recommend to use CDC in large data sets as dependence measure. 


4 Conclusion 

In this paper, we discussed the equitability and weak-equitability (power-equitability) of some common 
dependence measures, such as MIC, MI, dcor, CDC, ACE, RDC, HHG, Pearson correlation coefficient (pcor), 
Spearman rank correlation (scor), kendall’s r, HSIC. The equitability requires a dependence measure gives 
similar scores to equally noisy data sets no matter what kind of functional relationship is underlying them, 
this requirement is so strong that it is can not be satisfied by any dependence measure. Therefore, we 
give our definition of power-equitability, which only requires that all kinds of functional relationship, linear 
or non-linear, can be detected with the same possibility by dependence measure D at significant level a, 
when it is used in a independent test with the null hypothesis: x and y are independent, it is called also as 
weak-equitability because if D is equitable, then it is power-equitable. 

Self-equitable is meaningful and the basic requirement for a dependence measure used as a noise measure. 
However, a self-equitable measure, such as MI, may not have a higher testing power as shown in our simulation 
results (see Figure [5]). 

Based on our simulation, we find that MI is more equitable than MIC, and HHG is power-equitable, ACE 
and CDC is secondary to HHG. Despite the computation time, HHG is worthing highly recommendation in 
each kind of application. 
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Figure 1: (color online) Equitability of ACE and RDC with mean(MSNR)=11.529, sd(MSNR)=0.233, 
range(MSNR) is 10.678 to 12.471. The simulation is based on 21 types of relationship (Definitions are 
given in appendix), and MSNR is controlled using <©• In this simulation we take the sample x from uniform 
distribution with length 5000, and noise is sampled from standard normal distribution. We sample x 100 
times and get 100 values of dependence measure for each kind of functional type. According to the definition 
of equitability, if we get almost the same score given by dependence measure D to each kind of functional 
type, then D is equitable. In our simulation, it is equal to that if the lines with different colors in the figure 
are consistent, then D is equitable, we can see from the figure they are both not equitable. 
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Figure 2: (color online) Equitability of MIC, MI, CDC, pcor, kcor, scor with mean(MSNR)=T1.529, 
sd(MSNR)=0.233, range(MSNR) is 10.678 to 12.471. (Legends are the same as that in Figure 1.) The 
simulation is based on 21 types of relationship (Definitions are given in appendix), and MSNR is controlled 
using <©• In this simulation we take the sample x from uniform distribution with length 5000, and noise 
is sampled from standard normal distribution. We sample x 100 times and get 100 values of dependence 
measure for each kind of functional type. According to the definition of equitability, under these setting, if 
we get almost the same score given by dependence measure D to each kind of functional type, then D is 
equitable, we can see from the figure they are all not equitable. In our simulation, it is equal to that if the 
lines with different colors in the figure are consistent, then D is equitable. MI is equitable in a small set of 
functional types except four types: Sigmoid, Lopsided L-shaped, L-sliaped and Spike. MI is more equitable 
than MIC, since MI values of 17 functional types (except the four types mentioned before) are centered 
around 1.0 to 1.2 with smaller variance than that of MIC. 
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Figure 3: (color online) Equitability of RDC, ACE, MIC, MI, CDC, pcor, kcor, scor with 
mean(SSNR)=10.471, sd(SSNR)=0.209, range(SSNR) is 9.382 11.081. (Legends are the same as that in 
Figure 1.) The simulation is based on 21 types of relationship (Definitions are given in appendix), and SSNR 
is controlled by the heuristic algorithm 1 and eq. [7] In this simulation we take the sample x from uniform 
distribution with length 5000, and noise is sampled from standard normal distribution. We sample x 100 
times and get 100 values of dependence measure for each kind of functional type. According to the definition 
of equitability, under these setting, if we get almost the same score given by dependence measure D to each 
kind of functional type, then D is equitable, we can see from the figure they are all not equitable. In our 
simulation, it is equal to that if the lines with different colors in the figure are consistent, then D is equitable. 
MIC is equitable in a large set of functional types except four types: Exponential [2®], Lopsided L-shaped, 
L-shaped and Spike. MI is less equitable than MIC, since MIC values of 17 functional types (except the four 
types mentioned before) are centered around 0.8 with smaller variance than that of MI. 
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Figure 4: (color online) Weak-Equitable of ACE and RDC with mean MSNR range from 0.7509836 to 
3.0057, and standard variance of MSNRs range from 0.039 to 0.15947. (Legends are the same as that in 
Figure 1.) The simulation is based on 21 types of relationship (Definitions are given in appendix), and 
MSNR is controlled using ([5]). In this simulation we take the sample x from uniform distribution with length 
5000, and noise is sampled from standard normal distribution. The power was estimated from 300 times 
simulation at significant level a = 0.05. We can see from the results that ACE is much more equitable than 
RDC in that range of MSNR, here we emphasize on the range of MSNR because if there is a very small 
MSNR for the data, then all of the methods will have a very small power. 
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Figure 5: (color online) Weak-Equitable of MIC, MI, CDC, poor, kcor and scor with mean MSNR range from 
0.7509836 to 3.0057, and standard variance of MSNRs range from 0.039 to 0.15947. (Legends are the same 
as that in Figure 1.) The simulation is based on 21 types of relationship (Definitions are given in appendix), 
and MSNR is controlled using <§• In this simulation we take the sample x from uniform distribution with 
length 5000, and noise is sampled from standard normal distribution. The power was estimated from 300 
times simulation at significant level a = 0.05. We can see from the results that CDC is much more equitable 
than others in that range of MSNR, here we emphasize on the range of MSNR because if there is a very 
small MSNR for the data, then all of the methods will have a very small power. For MI and MIC, their 
power-equitable is very similar, however, it seems that MI is more powerful than MIC in these functional 
types. 
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Figure 6: (color online) Weak-Equitable of dcor, HHG, and HSIC with mean MSNR range from 0.7509836 
to 3.0057, and standard variance of MSNRs range from 0.039 to 0.15947. (Legends are the same as that in 
Figure 1.) The simulation is based on 21 types of relationship (Definitions are given in appendix), and MSNR 
is controlled using ©• In this simulation we take the sample x from uniform distribution with length 5000, 
and noise is sampled from standard normal distribution. The power was estimated from 300 times simulation 
at significant level a = 0.05. We can see from the results that HHG is much more equitable than others in 
that range of MSNR, here we emphasize on the range of MSNR because if there is a very small MSNR for 
the data, then all of the methods will have a very small power. 
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Figure 7: (color online) Weak-Equitable of ACE, RDC and dcor with mean SSNR range from 0.0496 to 3.282. 
(Legends are the same as that in Figure 1.) The simulation is based on 21 types of relationship (Definitions 
are given in appendix), and SSNR is controlled by the heuristic algorithm 1 and eq. [7] In this simulation we 
take the sample x from uniform distribution with length 5000, and noise is sampled from standard normal 
distribution. The power was estimated from 300 times simulation at significant level a = 0.05. We can see 
from the results that dcor is much more equitable than others in that range of SSNR. 
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Figure 8: (color online) Weak-Equitable of MIC, MI, CDC, poor, kcor and scor with mean SSNR range 
from 0.0496 to 3.282. (Legends are the same as that in Figure 1.) The simulation is based on 21 types of 
relationship (Definitions are given in appendix), and SSNR is controlled by the heuristic algorithm 1 and eq. 
[7J In this simulation we take the sample x from uniform distribution with length 5000, and noise is sampled 
from standard normal distribution. The power was estimated from 300 times simulation at significant level 
a = 0.05. We can see from the results that CDC and MIC are much more equitable than others in that range 
of SSNR. For MI and MIC, the power-equitable of MI is similar than that of MIC. 
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5 Appendix 

5.1 Definition of functions 

1. Line: y = x 

2. Linear-|-Periodic, Low Freq: y = 0.2sin(4(2x — 1)) + ^(2x — 1) 

3. Linear+Periodic, Medium Freq: y = sin(107rx) + x 

4. Linear+Periodic, High Freq: y = 0.1 sin(10.6(2x — 1)) + pj(2x — 1) 

5. Linear+Periodic, High Freq: y = 0.2 sin(10.6(2x — 1)) + y^(2x — 1) 

6 . Non-Fourier Freq [Low] Cosine: y = cos(77rx) 

7. Cosine, High Freq: y = cos(147rx) 

8 . Cubic: y = 4x 3 + x 2 — 4x 

9. Cubi, Y-stretched: y = 41 (4x 3 + x 2 — 4x) 

10. L-shaped: y = x/99/(x < ^) +/(x > ygg) 

11. Exponential [ 2 X ] : y = 2 X 

12. Exponential [10 s ] : y = 10 s 
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13. Parabola: y = 4a ; 2 

14. Non-Fourier Freq [Low] Sine: y = sin( 97 ra;) 

15. Sine, Low Freq: y = sin( 87 rx) 

16. Sine, High Freq: y = sin( 167 ra;) 

17. Sigmoid: y = [50(x - 0.5) + 0.5]J(^ < x < + I(x > 

18. Varying Freq [Medium] Cosine: y = sin( 57 rx(l + a:)) 

19. Varying Freq [Medium] Sine: y = sin( 67 ra;(l + x)) 

20. Spike: y = 20/(x < ^) + [-18x + < x < + [-§ + \]I(x > 

21. Lopsided L-shaped: y = 200a cl(x < + [-198a; + < x < ^ + [-§ + ^]/(x > 3 ^) 

where /(•) is the indicator function. 

5.2 Data for simulation 
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Linear+Periodic,Medium Freq SNR: 2.4 


Cosine, High Freq SNR: 2.78 


Cubic SNR: 2.96 



in 

o 

in 

i 


Linear+Periodic,Medium Freq SNR: 1.1 


Cosine, High Freq SNR: 1.05 


Cubic SNR: 1.66 




in 


in 

in 

i 


Exponential^0 A x] SNR: 2.22 


Varying Freq [Medium] Sine SNR: 2.39 


Lopsided L-shaped SNR: 1.47 



Exponential^0 A x] SNR: 1.3 


Varying Freq [Medium] Sine SNR: 1.25 


Lopsided L-shaped SNR: 1.08 



Figure 9: 6 types of relationships with different SNR are shown here. Definitions of relationships are given 
in appendix 5.1), and SSNR is controlled by the heuristic algorithm 1 and eq. [TJ All the simulations about 
SSNR were taken by such data. 
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